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Abstract 


Learner engagement and sustained motivation belong to some central challenges in designing and 
facilitating Massive Open Online Courses (MOOCs). One approach to face this challenge has been the 
use of gamification as an attempt to enhance learner engagement and motivation through the use of 
game-related elements such as levels, leaderboards, points and badges. While gamification has been 
successfully implemented in small-scale learning designs, research on gamification in MOOCs is still in 
its early stages. This paper addresses this research gap and presents a meaningful gamification 
approach applied to the design and facilitation of a series of eight mini-MOOCs in a learning environment 
called Open Virtual Mobility Learning Hub (OpenVM Hub). The paper examines learner experience in 
meaningfully gamified MOOCs in the OpenVM Learning Hub by analysing data from the evaluation 
survey answered by 945 learners. The results of the data analysis indicate that meaningful gamification 
can be considered a valuable method for designing engaging MOOCs, especially if it is combined with 
a modular and flexible mini-MOOC approach, which focuses on short learning events, light bits of 
learning content and a diversity of media formats used to support learning. 


Keywords: Gamification, meaningful gamification, Massive Open Online Courses, Open Virtual Mobility, 
Virtual Mobility. 


1 INTRODUCTION 


Learner engagement and sustained motivation belong to some central challenges in designing and 
facilitating Massive Open Online Courses (MOOCs). One approach to face this challenge has been the 
use of gamification as an attempt to enhance learner engagement and motivation through the use of 
game-related elements such as levels, leaderboards, points and badges. Gamification has been defined 
as using game elements in non-game contexts [1] and as a design approach based on game thinking, 
used to create better experiences [2]. While gamification has been successfully implemented in small- 
scale learning designs, research on gamification in MOOCs is still in its early stages. Also researchers 
have pointed out that the spectrum of game elements has not yet been fully explored in current 
gamification designs [3]. 


Moreover, given some of the limitations of traditional gamification approaches, e. g. scoring-based 
gamification or BLAP gamification (Badges, Leaderboards, Achievements, Points), new approaches 
focusing on meaningful engagement (meaningful gamification) and/or blending game-based and play- 
based elements (deep gamification) have emerged and started being applied in MOOC designs. 
Specifically, the meaningful gamification approach has addressed the problem of limited possibilities in 
traditional gamification for an individual to make choices without external control, which may have 
negative effects on self-regulation. While BLAP gamification focuses on the use of the gamification loop 
[4] with an award system based on points as the central element of gamification, meaningful gamification 
proposes to enhance learner engagement and motivation without emphasising external rewards. For 
example [5] [6] argue that integrating meaningful options into the gamification designs not only improves 
user experience but also learning outcomes. The concept of meaningful gamification builds on 
motivational research, which show that many forms of external rewards, such as point-based award 
systems in BLAP gamification offer only limited possibilities for an individual to make choices and may 
even lead to negative effects on self-regulation in the learning process [6] [7]. Until now, however, there 
is little research about the applications and effects of meaningful gamification in MOOC designs. In this 
paper we present research results about the effects of a meaningful gamification design in MOOCs 
developed in the European project Open Virtual Mobility. 
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2 MEANINGFUL GAMIFICATION APPROACH 


Open Virtual Mobility project (short: OpenVM)' is a three year (2017-2020) strategic partnership for 
innovation and the exchange of good practices founded by the European Erasmus+ program of the 
European Commission. The OpenVM Learning Hub? aims to create engaging and effective learner 
experience by making use of meaningful gamification as an approach to enhancing learner engagement 
in online learning environments. The OpenVM Learning Hub, an online learning environment based on 
LMS Moodle with integrated badging and e-portfolio systems, hosts a set of eight mini-MOOCs which 
support achievement, assessment and recognition of virtual mobility skills. Mini-MOOCSs are a type of 
MOOCs which are smaller, shorter, cover less content and fewer skills than traditional MOOCs [8]. 
OpenVM mini-MOOCs are designed following the principles of micro-learning and aim to provide 
alignment of micro-learning objectives, micro-learning activities and micro-learning assessment with 
micro-credentials based on the Open Badges metadata standard and issued upon successful 
completion of each of the three levels in a given mini-MOOC [9]. 


The OpenVM Learning Hub hosts altogether eight different mini- MOOCs each dedicated to a specific 
virtual mobility competency area. The eight skill areas were identified through a Group Concept Mapping 
research study [10]. The research results were used to define the skill set with eight main competency 
areas and respective sub-skills. Each of the eight competency areas can be developed at three levels, 
i. e. foundation, intermediate and advanced. OpenVM mini-MOOCs are designed to support learners in 
developing virtual mobility competencies in the eight areas which required for successful engagement 
in virtual mobility in higher education [11]. The eight competency areas are (1) Media and digital literacy, 
(2) Active self-regulated learning skills, (3) Autonomy-driven learning, (4) Networked learning, (5) 
Intercultural skills and attitude, (6) Interactive and collaborative learning in an authentic international 
environment, (7) Open-mindedness, and (8) Open virtual mobility knowledge. 


Each mini-MOOC addresses one of the eight OpenVM competency areas as learning outcomes and 
aligns micro-learning activities and e-assessment to support the achievement of these learning 
outcomes. The alignment of learning outcomes, learning activities and e-assessments has been 
supported by semantic competency definitions referenced in the competency directory [10]. Upon 
successful completion of each mini-MOOC at one of the three levels, a digital micro-credential (Open 
Badge) can be claimed by the learner to recognise the specific competency. Digital micro-credentials 
serve as evidence for achievement of the learning outcomes in the eight competency areas. The aim is 
to enhance the readiness of students to engage in virtual mobility and to leverage the MOOCs to prepare 
for the experience or as a support during such an experience. Digital micro-credentials are an important 
element of the meaningful gamification design as they aim to enhance meaningful engagement of 
learners with learning content and learning activities in OpenVM mini-MOOCs. 


The OpenVM approach to meaningful gamification aims to enable learner control and ownership of 
learning in alignment with the principles of Personal Learning Environments, PLEs [12] [13] [14]. The 


OpenVM meaningful gamification approach also build on the models of self-regulated learning, SRL 


[15]. SRL is characterised by active monitoring and regulation of learning processes such as setting and 
orientation towards learning goals, strategies used to achieve goals, management of resources and 
reactions to feedback in formative assessment [16]. The meaningful gamification approach implemented 
to the design of mini-MOOCs in the OpenVM Learning Hub aims to enhance self-regulation through 
providing transparent criteria for attainment, assessment and recognition of competencies, meaningful 
feedback and opportunities for self-assessment. Hence, OpenVM meaningful gamification approach, 


rooted in self-regulation and learner control, emphasises the importance of allowing learners to take 


decisions and make choices, e. g. allowing learners to choose their learning pathways to achieve own 
learning goals as well as encouraging learners to take responsibility for own learning in terms of goals, 
processes and outcomes [17]. 


Based on research showing that meaningful frames can be successfully combined with award systems 
[18], OpenVM gamification approach combines non-reward (e. g. choices) with reward-based methods 
(e. g. digital credentials) to help learners find meaning and relevance in learning activities and to allow 
learners to make decisions about learning goals, steps and activities. Based on the results of an online 
survey with 13 project partners in 2018, elements of meaningful gamification in the OpenVM project 
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were divided into ten groups, i. e. (1) goals, (2) content, (3) activities, (4) assessment, (5) progress, (6) 
feedback, (7) engagement, (8) collaboration, (9) personalisation and (10) support of learning, and 
implemented following a unified design for learning in each mini-MOOC [19]. 


3 METHODOLOGY 


The data presented in this paper was collected using the OpenVM Evaluation Questionnaire — an online 
survey designed by the project partners. Learners in the OpenVM Learning Hub are invited to fill in the 
questionnaire at the end of a MOOC. The results presented here come from the first pilot phase in the 
OpenVM project, which took place from September to December 2019. The first pilot-phase was 
intended to assess the quality of the OpenVM MOOCs and the main MOOC components including 
meaningful gamification elements. As sample of n=945 learners responded to the online survey. Fig. 1 
below shows which mini-MOOCs the respondents participated in. 


Which of the following MOOC did you attend and do you want to assess? 
945 Antworten 


@ Media and Digital Literacy 


8% |8,8% @ Intercultural Skills 
@ Active Self-Regulated Learning 
@ Autonomy-drive learning 
= | @ Networked Learning 


f @ Collaborative Learning 
@ Open-mindedness 
@ Open Education and Virtual Mobility 


Figure 1. Participation in OpenVM MOOCs. 


OpenVM Evaluation Questionnaire is organised in eight sections. In all eight sections participants are 
required to express their level of agreement with a set of statements related to specific MOOC design 
elements on a Likert scale from 1 (strongly disagree) to 5 (totally agree). OpenVM Evaluation 
Questionnaire was created using Google Formular and encompasses these eight sections: 


1 Personal details (such as age, gender, affiliation and role, in this section participants are required 
to say which of the eight mini-MOOCs they are assessing), 


2 Questions regarding the overall MOOC design (such as learning experience, quality of content 
instruction and support for learning), 


3 Questions regarding digital credentials and meaningful gamification (such as quality of design, 
motivation, engagement and possibilities of choice), 


Questions regarding technical aspects (such as use and usability), 


Questions regarding the foundation level of a mini-MOOC (such as duration, language, content, 
use of multimedia), 


6 Questions regarding the intermediate level of a mini-MOOC (extending questions from the 
foundation level by questions related to the matching tool and group formation activity, which are 
specific design elements used at this level), 


7 Questions regarding the advanced level of a mini-MOOC (extending questions from the 
foundation and intermediate levels by questions related to e-portfolio and peer-assessment 
activities, which are specific design elements at this level), 


8 Questions related to the investigation about the extent to which MOOCs supported self-regulated 
learning (these question were answered by a sample of n=562 learners, i. e. by 59% of all 
respondents who attended Active Self-Regulated Learning and/or Autonomous Learning 
MOOCs. 


In the section below we present selected results based on the data from the OpenVM Evaluation 
Questionnaire related to meaningful gamification design elements. 


9531 


4 RESULTS 


The data collected though the OpenVM Evaluation Questionnaire provides rich material for examining 
learner engagement in mini-MOOCs hosted in the OpenVM Learning Hub. The results are discussed 
in relation to the ten groups of meaningful gamification design elements described in section 2, i. e. (1) 
goals, (2) content, (3) activities, (4) assessment, (5) progress, (6) feedback, (7) engagement, (8) 
collaboration, (9) personalisation and (10) support of learning. To aggregate the results, we combined 
scores 4 and 5 (4=agree; 5=strongly agree), both expressing agreement of respondents. 


(1) Learning goals: From the perspective of meaningful gamification it was important to explore 
whether learners could engage in leaning by making choices, especially in relation to choosing their 
learning pathways to reach own learning goals. Over 57% participants agreed that they could choose 
their own learning pathway in a mini-MOOC they participated in. Also we explored whether meaningful 
gamification elements added to mini-MOOCs to support the understanding of learning goals were helpful 
for learners. Over 66% agreed that learning outcomes were described in an understandable and 
meaningful way. Over 58% agreed that the Course Overview module added at the beginning of each 
mini-MOOC level course was helpful for orientation and learning. Over 61% agreed that the information 
about learning objectives was helpful too. 


(2) Learning content: From the perspective of meaningful gamification we wanted to explore how 
learners evaluate and engage with micro-learning content in the mini-MOOCs. Over 63% agreed that 
learning content is accurate. Over 64% of respondents agreed that examples provided in the micro- 
learning content can be understood by all learners, independent of the background. Moreover, almost 
68% appreciated the opportunity to use different kinds of content including text, video and pictures. 
These positive results indicate that learners can meaningfully engage with learning content in MOOCs. 
We also asked questions related to the contribution of learning content in form of Open Educational 
Resources (OERs) to self regulated learning. Most respondent agreed that OER content (texts, videos, 
pictures) positively contributed to SRL, especially in relation to self-motivation and task-analysis (61%), 
self-observation (60%), self-control (59%), self-evaluation and self-reaction (58%). 


(3) Learning activities: A number of different questions were asked about learning activities. From the 
perspective of meaningful gamification it was relevant to explore whether learners perceive learning 
activities as meaningful. Over 54% of learners agreed that learning in mini- MOOCs was meaningful to 
them. Over 64% agreed that learning activities provided support for self-regulation related to own 
learning path. Some learning activities like the e-portfolio activity were less appreciated by the 
respondents, e. g. less than 50% agreed that the e-portfolio activity supported their self regulated 
learning in relation to task analysis and self-observation (44%), self-evaluation (43%) and self-motivation 
(42%). 


(4) Assessment: From the perspective of meaningful gamification design we explored how learners 
viewed e-assessment opportunities created in OpenVM mini-MOOCs to self-assess and peer-assess 
virtual mobility competencies. The results show that e-assessment opportunities in mini-MOOCs 
enhanced self-regulated learning, especially in relation to self-observation (60%), self-evaluation (59%), 
task analysis (58%) and self-motivation (56%). 


(5) Learning progress: To improve self-monitoring of the learning progress as part of self-regulated 
learning, a progress bar using a customised version of the Moodle plugin “Mycourse status” was 
implemented in all mini-MOOCs. Over 58% of respondent agreed that this gamification element was 
useful for orientation about own learning progress. 


(6) Feedback: Self regulated learning may be supported by delivering high quality feedback. 
Gamification elements used to support meaningful feedback include possibilities to receive and to give 
meaningful feedback through peer-assessment, and through the used of tools to support self-monitoring 
of learning. Tools for self-monitoring include digital credentials (OpenVM badges), which provide 
feedback to learners about the state of their competency development and MOOC completion. Most 
respondents agreed that OpenVM badges supported them in self regulated learning, especially in 
relation to self-motivation (58%), self-observation and self-evaluation (56%), task analysis (55%), self- 
control (54%) and self-reaction (52%). Over 53% agreed that OpenVM credentials motivated them to 
learn in a mini-MOOC. Almost 58% agreed that OpenVM credentials motivated them to complete a mini- 
MOOC level. 


(7) Learner engagement: Over 60% of respondents agreed that OpenVM MOOCs are interactive and 
engaging. Over 55% agreed they had fun learning in mini-MOOCs. Over 62% agreed that the learning 
environment was friendly and usable. These results indicate that positive learner engagement, one the 
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key goals of meaningful gamification design, was experienced by a substantial group of learners in 
OpenVM mini-MOOCs. 


(8) Collaboration: In relation to collaboration we asked questions about the support of communication 
and peer-assessment for self regulated learning. Only a smaller group of learners agreed that 
communication forums and peer-assessment enhanced their self-regulated learning. For example, only 
44% of respondents agreed that peer-assessment enhanced their self-observation. Also, only 44% of 
respondents agreed that communication forums contributed to their self-motivation. 


(9) Personalisation: Meaningful gamification design in the OpenVM Learning Hub combines the 
principles of Personal Learning Environments (PLE) with Self-Regulated Learning (SRL). From the 
perspective of the PLE approach it is crucial that learners can take control of their learning by making 
personal choices. Personal choices are viewed as a mechanism for personalisation. Over 55% 
respondents agreed that they could choose different elements to learn in a mini-MOOCs indicating a 
high level of personal choice. Over 63% agreed that the possibility to take choices and deepen their 
learning where they wanted was very helpful for their learning. 


(10) Support of learning: Meaningful gamification approach emphasises the importance of support. In 
relation to support of learning we asked questions related to self regulated learning. The results show 
that especially the visualisation of learning pathways, progress, objectives, materials, times and 
competencies enhanced learner experience and SRL. A large group of learners agreed that visualisation 
elements supported their task analysis and self-evaluation (60%), self-motivation (61%), self-control 
(62%) and self-observation (63%). 


5 CONCLUSIONS 


The results described above indicate that the current meaningful gamification design of OpenVM 
MOOCs based on both PLE (choices, control) and SRL (analysis, motivation, observation) principles, 
has led to positive learner engagement. The results indicate that especially five design elements, i. e. 
(a) multimedia OERs, (b) flexible learning pathways, (c) opportunities for self-assessment, (d) support 
of learning and (e) digital credentials (Open Badges), have most positively impacted learner 
engagement in OpenVM MOOCs. Elements such as_ e-portfolios, peer-assessments and 
communication forums have been viewed as less helpful for learning. 


Further research should take a closer look at how to design more meaningful and engaging collaboration 
and peer-interaction opportunities in MOOCs. Also, we recommended to develop a set of standardised 
items for online-surveys to allow learners assess meaningful gamification design in a more 
comprehensive way. Further research should also combine other sources of data including qualitative 
data (e. g. from interviews) and learning analytics (e. g. engagement with specific design elements in 
MOOCs) to provide a more comprehensive picture of how learner engagement can be enhanced by 
gamifying MOOCs in a meaningful way. 
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